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Abstract 


This  research  addresses  the  implementation  of  am  electronic  element, 
which  emulates  the  biological  synaptic  interconnection,  in  an  artifidal 
electronic  neural  system.  The  basic  interconnection,  or  the  weight,  consists 
of  an  electrically  reprogrammable,  nonvolatile,  analog  conductance  which 
programs  at  5V  levels.  In  addition,  the  fabrication  technology  for  this 
synaptic  interconnection  is  compatible  with  existing  CMOS  VLSI  process. 
The  attractive  features  of  this  synaptic  weight  will  be  discussed  in  this 
report.  Furthermore,  this  report  examines  the  material  needs,  the  device 
structures,  the  izse  of  the  synaptic  weights  in  a  two-tap  weight  hnear 
adaptive  neiiral-Hke  circuit  and  the  issue  of  integrating  both  the  synaptic 
weight  elements  and  the  peripheral  circuit  onto  a  single  sihcon  wafer. 


1.  Introduction 

The  current  surge  of  enthusiasm  for  neural  network  aims  to  construct  systems  that  can  learn 
or  modify  their  behavior  according  to  the  environment.  There  are  many  similarities  which  exist 
between  this  new  class  of  machine  and  human  beings.  One  of  these  similarities  is  the  massive 
parallehsm  in  processing  information.  Parallel  processing^  concepts  are  in  stark  contrast  to  the 
operations  of  modem  digital  computers  that  perform  large  ntimbers  of  sequential  operations  very 
rapidly  and  accurately. 

Researchers  beheve  the  synaptic  junctions  in  a  nexiral  system  are  the  local  memory  sites 
and  provide  the  physiological  basis  for  the  distributed  parallel  systems.^'  ^  These  synapses  are  not 
only  modifiable  but  also  serve  the  functions  of  storing  and  transmitting  information  from  neuron  to 
neuron.  To  reduce  the  complex  modelling  required  for  the  synaptic  interconnection,  the 
representation  of  the  synapse  has  been  simplified  to  a  single  ideal  junction  between  the  output  of 
neurons  laxons)  and  the  inputs  to  neurons  idendrites).  Synaptic  modification  requires  information 
from  the  input  and  the  output  of  the  neuron  in  order  to  perform  complex  recognition.  Therefore,  the 
nature  of  the  synaptic  jimction  and  the  principle  or  algorithm  which  controls  local  organization  at 
the  neuron  level  become  two  central  issues  pertaining  to  neural  networks  research. 

The  recent  interest  in  neural  networks’’^  is  a  direct  consequence  of  the  programmability 
which  is  an  essential  feature  of  learning  machines,  associative  memories,  and  adaptive  signal 
processors.  Programmability  requires  a  modification  of  the  synaptic  strength  in  the  language  of 
neurobiology'.  If  we  seek  an  efficient  hardware  implementation  of  electronic  neural  systems,  then 
the  synapses  -  as  well  as  the  network  itself-  should  be  analog.  Several  attempts  have  been  made  to 
reahze  programmable  synapses,  either  digitally®  or  with  temporary  storage  on  the  input  capacitance 
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of  a  MOS  Transistor' '  ®  to  alter  the  latter’s  analog  conductance.  The  former  approach  stores  the 
weight  information  in  digital  registers  and  thus  suffers  from  excessive  chip  area  and  power 
consumption.  On  the  other  hand,  although  the  MOS  Transistor  provides  an  analog  synaptic 
strength  (weight)  in  a  small  chip  area,  the  weight  is  temporary  and  requires  periodic  refres’n  similar 
to  a  DRAM.  Thus,  this  dynamic  refresh  approach  lacks  the  nonvolatility  and  storage  properties  of 
an  EEPROM  cell.  Researchers  at  Intel  have  reported  an  electrically  trainable  artificial  neural 
network  with  floating  gate  device  as  the  synaptic  element.®  Although  floating  gate  device  has  the 
property  of  nonvolatility,  its  high  programming  voltage  requirement  prevents  it  from  being 
tec'nnologically  compatible  with  scaled  CMOS  process. 

In  this  research  report  we  descri’ne  a  new  approach  to  obtain  an  electrically  reprogrammable  or 
modifiable  synaptic  weight  to  be  used  as  a  basic  functional  element  in  electronic  neural  sv'stems. 
The  salient  features  of  this  network  element  are  the  following: 

•  Low  programming  voltages(5-10V)  which  are  compatible  with  peripheral 
CMOS  VLSI  technology  in  contrast  with  Floating  Gate  approaches. 

•  Low  power  dissipation  ( <  1  uW). 

•  Dynamic  Range  of  1000:1  (60  dB). 

•  Nonvolatile  features  which  mimic  biological  synapses  with  respect  to  memory 
loss  (e.g.  20%  of  the  information  available  after  10  years)  and  reinforced 
learning  (e.g.  successive  interrogation  enhances  memory  retention). 

•  Small  synaptic  area  on  a  VLSI  chip  (e.g.  less  then  20um“  for  1.25  um  feature 
sizes). 

•  Extensive  erase/write  programming  cycles  are  possible  with  this  synapse  ( > 

10®  cycles)  in  contrast  with  Floating  Gate  approaches. 

•  Inherent  radiation  damage  resistance  beyond  a  total  dosage  of  IMRad  (Co®®) 
and  lOP  Rad/sec  transient  which  is  not  possible  with  Floating  Gate  technology. 

Thus,  if  radiation  damage  resistance  of  neural  networks  is  an  important  issue, 
then  the  SONOS  devices  have  demonstrated  success  in  this  area. 

The  basic  nonvolatile  device  structure,  which  we  describe  in  this  report  was  first  introduced  as 
a  digital  nonvolatile  memory  cell  in  the  summer  of  1987  at  the  IEEE  Device  Research  Com'erence*® 
by  researchers  at  Lehigh  University.  We  have  had  a  continual  involvement  over  a  20  year  period 
with  nonvolatile  memories,  beginning  in  the  late  oO’s  where  we  had  programming  voltages  of  25^,'.  to 
the  late  30’s  with  our  novel  5\'  SONOS  detdce  structures.  During  this  time  period  we  introduced  the 
use  of  CCD’s  and  nonvolatile  memories^"' in  nonvolatile  charge  addressed  memones 
NOVCAM).  These  ideas  have  been  employed  recently  for  neural  networ’n  circuits  oy  researchers  at 


Lincoln  Laboratories.^^  Our  recent  work  recognizes  the  inherent  analog  conductance  aspect  of  the 
nonvolatile  SONOS  memory  device  which  makes  it  a  perfect  candidate  for  the  modifiable  synapse  in 
an  electronic  neural  system. 

In  addition  to  the  realization  of  an  electronic  element  to  simulate  the  synaptic  interconnections 
of  a  neural  network,  we  mtist  have  a  method  or  algorithm  to  change  or  reprogram  these 
interconnections  and,  thus,  alter  the  connectivity  of  the  netaral  network.  We  have  had  experience 
with  a  particular  form  of  an  algorithm,  namely,  the  Wldrow-Hoff  Least  Mean  Square  (LMS)^'’  error 
algorithm  or  in  neural  network  terminology  -  the  so-called  ’delta  rule’.  In  the  late  70’s  we  researched 
a  CCD  Adaptive  Analog  Signal  Processor^®-  which  realizes  the  'delta  rule’  with  CCD  analog  delay 
lines  and  electrically  reprogrammable  MNOS  analog  conductance  weights.  'These  weights  were 
nonvolatile  memory  transistors  whose  analog  conductance  was  programmed  with  voltages  ranging 
from  15-25V.  Our  recent  work  on  ’scaling’  these  programmable  analog  conductances  has  resulted  in 
a  new  device  structure,  called  the  SONOS  nonvolatile  memory  transistor,  which  can  be 
reprogrammed  with  voltages  ranging  from  5-lOV.  This  work  has  recently  been  described  at  the  1S91 
11th  IEEE  Nonvolatile  Semiconductor  Memory  Workshop.^®  These  voltage  levels  are  compatible 
with  ’scaled’  CMOS  VLSI  technologv’  which  has  12-15V  breakdo%vn  voltages  for  1.25um  feature  sizes. 
In  this  report  we  describe  our  recent  work  on  the  electrically  reprogramma’Dle  'modifiable)  SONCS 
nonvolatile  synapse  and  a  simple  electronic  neuron  with  2  synaptic  weights.  We  discuss  this  two-tap 
weight  linear  adaptive  neuron  in  terms  of  the  technologv',  the  electrical  characteristics  of  the 
synapses,  and  their  performance  in  this  simple  test  vehicle  -  a  'delta  rule’  adaptive  signal  processor. 

2.  Technology  and  Characterization  of  SONOS  Synaptic  Weight 

The  programmable  synapse  is  the  result  of  an  ongoing  effort  at  Lehigh  University-  to  als’  the 
programming  voltages  required  to  alter  the  analog  conductance  of  a  nonvolatile  memotv  transistor 
with  a  multi-layer  (.oxide- nitride-oxide.'  gate  insulator  as  shown  in  Fig.  1.  Recent  ■  ..Torts  in  scaling 
this  device  have  resulted  in  a  SONOS  (SihconBlocking  Oxide'Nitnde'Tunne'ung  Oxide  Silicon 
nonvolatile  memory  transistor  which  is  electrically  reprogrammable  at  CMOS  voltage  levels. 
Typically,  the  tunneling  oxide  is  15-25X  ,  the  storage  nitride  is  50-10P'.\  and  the  blocking  oxide  is 
35-oOX  .  Fig.2  s’nows  the  Transmission  Electron  Microscope  'TEM)  pnotograph  of  the  cross  sectional 
view  of  the  SONOS  transistor.  This  device  is  similar  to  a  SNOS  transistor  except  for  the  addihon  of 
the  blocking  oxide  whic’n  is  used  to  inhibit  injection  of  earners  from  the  polysilicon  gate  electrode 
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and  also  to  improve  the  memory  retention  by  prohibiting  the  transfer  of  stored  charge  from  the 
nitride  to  the  gate  electrode.  As  the  result,  the  blocking  oxide  permits  the  entire  dielectric  sandwich 
to  be  scaled  to  dimensions  where  the  programming  voltages  ranging  from  5-10  V  are  possible. 


When  the  SONOS  device  is  subjected  to  a  positive  (or  negative)  programming  pulse,  electrons 
(or  holes)  are  injected  into  the  silicon  nitride  layer  by  means  of  tunneling  across  the  thin  tunnel 
oxide.  The  injected  charges  are  trapped  by  the  silicon  nitride  and  thus  shift  the  threshold  voltage 
positively  (or  negatively).  The  threshold  voltage  of  a  SONOS  transistor  can  be  written  as 


V^H  - 


^ob 


)  Qn  +  -Ob 


where  4*3  is  the  bulk  potential,  OQg  is  the  gate  to  semiconductor  workfunction,  is  the  fixed  charge 
at  the  tunneling  oxide-sihcon  interface,  and  are  the  dielectric  permittivities  of  the  oxide  and 
nitride,  is  the  dielectric  nermittivitv  of  the  bulk  silicon,  x„,  is  the  tunnel  oxide  thickness,  x  •  is  the 
blocking  oxide  thickness,  is  the  nitride  thickness,  xis  the  charge  centroid  in  the  insulator,  and  Q>; 
is  the  charge  stored  in  the  nitride.  Xg  is  the  bulk  doping  density,  and 
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We  assume  that  the  ttinnel  oxide  and  blocking  oxide  have  the  same  dielectric  permittiviu-,  even 
though,  it  is  known  that  the  tunnel  oxide  is  silicon  rich  and  the  blocking  oxide  is  an  oxynitride.  The 
values  of  the  charge  centroid  x  and  the  charge  stored  in  the  nitride  will  change  as  the  device  is 
written  or  erased.  The  analog  conductance  of  the  SONOS  synaptic  weight  may  be  written  as 

_  W  ,  ,  , 

Sds- ^eSJI  ^eS^^GS~ ■' -  ' 


where  u^g.  is  the  effective  carrier  mobihtv'.  is  the  read  voltage,  and  is  the  electrically 
modifiable  threshold  voltage  given  in  equation  (1).  Therefore,  there  are  two  ways  which  the  analog 
channel  conductance  can  be  altered;  d)  change  the  value  of  or  (2)  change  the  value  of  In 
our  study,  the  latter  approach  is  chosen. 

The  SONOS  transistors  are  characterized  for  their  memory  properties  by  using  the  test  station 
described  in  Anirban  Roy's  Master's  Thesis.^®  This  test  station  allows  one  to  take  both  the 
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erase/write  and  retention  measurements.  To  investigate  the  memory  loss/retention  properties  of  the 
synaptic  weight  element,  retention  measurements  are  taken.  The  retention  characteristics  are 
obtained  by  applying  positive  (negative)  five  volts  to  the  gate  for  10  seconds  to  place  the  device  in  the 
write  (erase)  state  and  then  measuring  the  tum-on  voltage  after  a  varying  delay  time.  The  tum-on 
voltage  is  related  to  the  threshold  voltage  by 


i~hs 

Vj  =  +  'J— p— 


(4) 


with  Ijjg  as  the  forced  drain  to  source  current  dxiring  measurement  and 
P  ~  Peff  (■£  )  *^e£F 


where  W  is  the  width  of  the  transistor.  L  is  the  length  of  the  transistor,  and  Ugg-  is  the  effective 
mobility.  The  effective  mobility  is  the  bulk  mobility  reduced  by  Coulombic  and  surface  scattering  of 
carriers  in  the  inversion  layer.  This  mobility  is  influenced  by  the  gate  and  substrate  voltages.^®  For 
a  SONOS  transistor  retention  measurements  indicate  that  greater  than  20  percent  of  the  memory' 
window  remains  after  a  projected  10  year  delay  time  as  shown  in  Fig.  3.  The  erase-write 
measurements  indicate  the  progtamming  speed  of  the  synaptic  weight  element.  To  measure  the 
writing  (erasing)  speed,  negative  (positive)  five  volts  are  applied  to  the  gate  for  10  seconds  to  place 
the  device  in  the  erase  (write)  state.  Then,  positive  (negative)  five  volts  cire  appUed  to  the  gate  '.vith 
varying  pulse  widths  and  the  tum-on  voltage  is  measured  after  each  pulse  width.  The  erasewrite 
characteristics  of  the  SONOS  memory'  transistor  are  shown  in  Fig.  4.  A  wide  dynamic  range  is  one 
of  the  essential  properties  for  the  synaptic  weight  element,  and  Fig.  5  illustrates  a  60  dE  in  dynamic 
range  after  r5V  programming  for  the  SONOS  synaptic  weight.  In  addition,  a  recent  study  in 
reliability  has  demonstrated  the  inherent  resistance  of  the  SONOS  memory'  transistor  to  radiation 
damage  =  O.IV*.  with  V^g  =  +  fV  at  IMRad  Co'’®  radiation 

3.  Single-level  Linear  Adaptive  Neuron 

We  have  incorporated  the  SONOS  smaptic  weights  -mto  a  single-level  linear  neuron-like 
crcuit  using  a  Widrow-Hoffs  delta  learning  ruie.*^  The  circuit  is  built  with  a  hybrid  breadboard  of 
CMOS  components  for  the  control  logic  and  the  algorithm  implementation  and  the  SONCS 
nonvolatile  memory  transistors  to  demonstrate  the  voltage  level  compatibility  of  both  SONCS  and 
CMOS  technologies.  Many  researchers  believe  that  the  neural  system  is  made  up  of  several  layers' 
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of  neurons  and  Fig.  6  shows  the  multi-layer  architecture  of  an  artificial  neural  network.  The  first 
layer  of  neurons,  the  input  layer,  can  be  best  thought  as  the  sensory  neurons  in  a  human  body.  The 
weight  connections  between  the  input  layer  and  the  middle  hidden  layer  are  normally  considered  to 
be  feedforward  and  fixed.  On  the  other  hand,  the  weight  connections  between  the  middle  hidden 
layer  and  the  output  layer  are  considered  to  be  feedback  in  nature.  Our  work  has  concentrated  on 
the  implementation  of  two  neurons  in  the  hidden  layer  and  one  output  neuron  as  highhghted  in  the 
figure. 

Fig.  7  shows  the  block  diagram  of  the  single-level  Hnear  adaptive  neuron.  A  desired  response 
for  external  teacher),  d(m),  is  presented  to  the  neuron  as  the  training  signal.  If  the  output  of  the 
hnear  adaptive  neuron  is  not  trained,  then  there  exists  a  mismatch  between  the  output  of  the  linear 
adaptive  neuron,  y{m),  and  the  desired  response,  d{m). 

e(/n')  =  dim)  —  yim)  <6) 

where  <.[m)  is  the  error  generated.  This  error  is  then  used  by  a  learning  algorithm,  namely  the 
Chpped-data  Least  Mean  Error  algorithm,  to  minimize  the  error  generated  and  thereby  training  the 
neuron  to  the  correct  response.  This  single-level  linear  adaptive  neuron  has  two  tap  weights,  each 
weight  is  composed  of  two  SCNOS  analog  electrically  reprogrammable  conductances  as  shown  in 
Fig.  8.  Since  the  synaptic  weight  must  be  either  positive  or  negative  in  value,  we  have  chosen  a 
differential  weighting  scheme.  If  the  analog  conductance  coimecting  the  positive  summing  path  to 
the  differential  operational  amplifier  is  greater  than  the  analog  conductance  connecting  the  negative 
summing  path  to  the  differencial  operational  amplifier,  then  the  weight  is  positive  in  value.  On  the 
other  hand,  if  the  opposite  case  is  true,  then  the  weight  is  negative  in  value.  Positive  weight  value 
corresponds  to  the  excitatory  synaptic  strength  and  the  negative  weight  value  corresponds  to  the 
inhibitory  svmaptic  strength. 

In  operation,  the  input  signal  .nr)  is  passed  through  a  switched  capacitor  analog  delay  line 
where  the  input  signal  is  sampled  and  delayed  to  create  two  tapped  signal  outputs  and 
These  tapped  signals  multiply  m  their  corresponding  programmable  weights  '.V,.,  and  iV.  and  the 
result  is  summed  Uneariy  at  the  summing  amplifier.  The  output  ymi  can  be  expressed  as; 

1 

vimi  =  y  W.xmVx _ .  T'* 
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where  m  is  the  time  index  and  k  is  the  spatial  index.  A  correlated  double  sampling  technique“^s 
employed  in  the  circuit  to  remove  the  unwanted  noise  and  offset  voltages  introduced  by  the  summing 
amplifiers.  The  linesir  adaptive  neuron  is  configured  to  perform  Widrow-Hoffs  delta  rale  as: 

W^{m+l)  =  Wfim)  +  A  (8) 

where  A  W(m)  is  the  incremental  weight  to  be  calculated  by  the  clipped-data  least  mean  square  error 
(C-LMSE)  algorithm^: 

A  W^(m)  =  2a  |£(m)|  Sgnleim)]  Sgn[x(m-k)]  (9) 

where  u  is  the  convergence  factor.  Compared  to  the  regular  Least  Mean  Square  Error  algorithm,  the 
input  signal  amplitxide  is  clipped  in  the  learning  algorithm.  This  algorithm  eliminates  the  usage  of  a 
four  quadrant  multiplier  needed  for  the  LMS  error  algorithm.  The  sign  midtiplication  in  the 
incremental  weight  calculation  is  essentially  an  Exclusive  OR  operation  and  the  output  of  the 
Exclusive  OR  gate  controls  the  path  of  proper  gate  programming  voltage  for  the  SONOS  synaptic 
weight.  If  the  convergence  factor  is  small,  then  the  sy'stem  will  minimize  the  misadjustment  caused 
by  the  variance  of  the  weights;  however,  this  also  results  in  a  long  convergence  time.  Conversely,  if 
we  choose  to  use  a  larger  convergence  factor,  then  the  convergence  time  of  the  system  is  shortened 
with  the  penalty  of  larger  nusadjustment.  The  backpropagating  error  is  used  to  calculate  the 
adjustments  to  minimize  the  system  error  as  shown  in  equation  (9).  Once  the  error  is  minimized, 
the  system  is  saud  to  be  in  its  steady  state  condition"^  where  the  output  of  the  system.  y[m!.  is  the 
best  match  of  the  training  signal,  dim),  or  the  ’external  teacher’. 

The  incremental  weight  update  is  essentially  a  cross  correlation  between  the  error  and  the 
c’lipped  input  data  vectors.  The  update  stops  when  the  two  vectors  become  orthogonal.  Sometimes, 
t'ne  network  may  be  overccrrected  imcially.  'Powever.  the  error  will  be  quickly  minimized  by  the 
learning  algorithm  and  the  system  reaches  its  desired  response.  The  digital  delay  hne  pro^•ides  the 
sign  information  of  the  input  to  the  learning  algorithm.  A  special  steering  network  is  designed  to 
switch  the  proper  programming  voltages  to  the  gate  terminals  of  the  SONOS  transistors  once  the 
incremental  weig'nts  are  calculated. 


4.  Experimental  Resiilts 

There  are  two  main  types  of  characteristics  from  which  the  electrical  performance  of  the  linear 
adaptive  neuron  can  be  evalimted.  The  first  characteristic,  namely  the  output  and  training  signals 
versus  time  characteristics,  gives  the  information  on  how  well  the  output  signal  approximates  the 
training  signal  especially  in  the  phase  relationship  between  these  two  signals.  The  second 
characteristic,  namely  the  error  signal  versus  time  characteristics,  shows  how  fast  the  linear 
adaptive  neuron  adapts  before  it  reaches  its  minimum  error.  A  typical  output  and  training  signals 
versus  time  characteristic  consists  of  two  parts:  the  initialized  and  the  adapted  part.  In  the 
initialized  part,  the  weights  are  first  initialized  to  a  known  state  (either  the  fully  positive  or  the  fully 
negative  state)  and  then  the  weights  are  subjected  to  a  reading  voltage  to  read  out  the  weight 
information  and  the  output  signal  and  the  training  signal  are  compared  and  recorded.  The  Linear 
adaptive  neuron  is  then  allowed  to  adapt  itself  to  the  training  signal  and  the  results  are  sho%vn  in 
the  adapted  part  of  the  characteristics.  Figure  9  shows  the  output  and  training  signal  versus  time 
characteristic. 

A  typical  error  signal  versus  time  characteristic  is  obtained  'vith  initialized  weight  values  and 
monitoring  the  error  signal  %vith  time.  Our  observation  indicates  the  weight  irdnahzation  scheme 
affects  the  convergence  behavior  of  the  linear  adaptive  neuron.  This  phenomenon  is  attributed  to 
the  nonsymmetric  erase  and  write  characteristics  of  the  SONOS  transistor.  Therefore,  one  weight 
initialization  scheme  may  require  more  erase  action  taking  place  than  another  weight  initialization 
scheme,  causing  a  difference  in  convergence  characteristics.  Figures  10  shows  a  typical  error  versus 
time  characteristic. 

5.  Technical  Progress 

The  performance  of  the  linear  adaptive  neuron  depends  strongly  on  the  programming 
characteristics  of  the  synaptic  weight  elements.  Therefore,  a  fabrication  run  aimed  specifically  at 
improving  the  programming  speed  of  the  symaptic  weight  elements  has  been  completed  and  fully 
char  cterized.  The  main  differences  between  the  new  and  the  old  devices  are  the  thicknesses  of  the 
storage  nitride  layer  and  the  blocking  oxide  layer.  By  reducing  the  nitride  layer  and  increasing  the 
blocking  oxide  layer  thicknesses,  the  gate  injection  of  earners  is  minimized.  Furthermore,  the 
charge  tunneled  across  the  tunneling  oxide  is  better  retained  m  the  nitnde  layer  with  a  thicker 
blocking  oxide.  As  a  result,  the  programming  speed,  indicated  by  the  erase-write  characterization  of 
the  new  devices,  demonstrates  a  roughly  one  order  of  magnitude  improvement  in  the  programming 
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speed  (determined  by  the  cross-over  time  as  mentioned  in  the  previous  section)  over  the  old  de’/ices 
as  shown  in  figrire  11.  In  addition,  the  programming  voltage  dependence  on  the  programming  speed, 
as  shown  in  figure  11,  illustrates  an  improvement  of  one  order  of  magnitude  in  programming  speed 
with  each  one  volt  increment  in  the  programming  voltage.  We  have  also  incorporated  the  new 
synaptic  weight  elements  into  the  linear  adaptive  neuron.  Since  the  programming  speed  of  the  new 
synaptic  weight  elements  has  improved  by  a  factor  of  10,  the  convergence  time  (a  measure  of  the 
performance  of  the  linear  adaptive  neuron)  has  been  reduced  by  a  factor  of  10  accordingly  as  shown 
in  figure  12. 

A  fully  computer  controlled  data  acquisition  sj^stem  is  an  invaluable  tool  for  synaptic  weight 
element  characterization.  The  current  measurement  system  requires  the  operator  to  manually  set 
up  the  measurement  sequence  and  hand-record  the  data  obtained.  An  automatic  data  acquisition 
system  enables  the  user  to  set  up  the  measurements,  and  analj’ze  the  data,  all  under  the  control  of 
one  console.  The  block  diagram  of  such  a  system  is  shown  in  figure  13.  A  vital  part  of  the  proposed 
system,  the  HPIB  command/data  interpreter  has  been  successfully  designed,  constructed,  and  fully 
tested  for  functionahty.  The  schematic  of  the  HPIB  command/data  interpreter  is  shown  in  figure  14. 

We  have  continued  our  efforts  on  the  integration  of  the  linear  adaptive  neuron  into  a  smgle 
silicon  wafer.  We  have  layed  out  some  of  the  vital  parts  of  the  linear  adaptive  neuron  under  the 
Mentor  Graphics  package  vrith  a  technology  specifically  geared  to  the  fabrication  process  of  our 
microelectronics  laboratory  at  Lehigh.  Since  we  are  creating  analog  ASICs,  area  and  power 
consumption  of  the  designs  have  to  be  minimized.  Upon  completion  of  layout,  an  extraction  of  the 
netlist  and  the  parasitics  are  obtained  and  SPICE  circuit  simulations  are  performed  to  ensure  the 
designs  are  properly  transferred  into  layouts.  Figures  15  and  16  illtistrate  a  sample  of  the  layouts 
already  completed. 

The  research  efforts  during  the  period  of  March  1991  to  September  1991  have  resulted  in  a 
paper  to  be  presented  at  the  International  Jomt  Conference  on  Neural  Networks  TJCNN  91  to  be 
held  in  November  at  Singapore. 
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6.  Proposed  Investigations 

We  have  demonstrated  the  direct  relationship  between  the  SONOS  synaptic  weight  element 
characteristics  and  the  performance  of  the  linear  adaptive  neuron.  Therefore,  we  feel  it  is  important 
tf/  continue  our  research  efforts  on  the  synaptic  weight  elements.  In  particular,  the  investigation  of 
how  to  make  even  faster  devices  has  resulted  in  the  fabrication  plans  of  utilmng  thinner  tunneling 
oxide  thickness  and  p+  polysilicon  gate  material. 

We  have  also  pledged  our  effort  to  the  development  of  the  automatic  data  acquisition  system  to 
further  aid  our  research.  A  customized  computer  program  will  be  written  to  drive  and  interface  with 
the  existing  measurement  equipment.  The  data  obtained  can  then  be  collected,  formatted,  and 
analyzed  with  device  parameter  extraction  routines. 

We  will  focus  on  the  layout  design  of  the  remaining  parts  of  the  Hnear  adaptive  neuron.  We 
believe  the  advent  in  SONOS  device  technology’  and  the  integration  of  the  signal  processing  circuitry 
as  well  as  the  synaptic  weight  element  onto  a  single  silicon  wafer  will  make  an  impact  to  the 
Artificial  Intelligence  Neural  Network  Technology  field. 

7.  Conclusions 

The  SONOS  nonvolatile  memory  transistor  has  been  shown  to  be  an  ideal  electronic  element 
for  the  electrically  reprogrammable  analog  conductance  in  an  artificial  neural  network.  We  have 
demonstrated  the  attractive  features  of  this  synaptic  weight  for  the  use  of  large  neural  network 
systems,  for  instance,  low  programming  voltage  (5-lOV),  low  power  dissipation(<lpW  /  synapse';, 
small  chip  area  (estimated  20um~/  weight  cell  for  a  1.2  pm  featxire  size),  a  dynamic  range  of  60  dB. 
good  memory  retention  (20  9o  window  at  a  projected  10  years  period),  and  endurance  beyond  10' 
erase'write  cycles.  In  addition,  the  SONOS  synaptic  weight  has  inherent  resistance  to  radiation 
damage  (AV'^  =  0.n',  with  =  at  IMRad  Co®®  radiation).  We  have  been  continuing  our  efforts  in 
optimizing  the  modifiable  synaptic  weights  to  provide  better  electrical  characteristics  for  neural 
network  applications. 

We  have  also  incoiporated  the  SONOS  synaptic  weights  into  a  single-level  two  tap  linear 
adaptive  neuron  employing  a  Widrow-HoiTs  delta  learning  rule.  The  combination  of  CMOS  control 
circuits  and  SONOS  synaptic  weights  has  demonstrated  the  feasibility  of  integrating  these  r.vo 
technologies  onto  a  single  silicon  wafer.  The  initial  results  are  encouraging  and  promising  and 
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provide  insight  and  direction  into  the  integration  of  these  two  technologies  to  realize  large  artificia] 
neural  network  systems. 
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Figure  1.  Cross  Sectional  View  of  the  (a)  MNOS  <h)  SONOS  Electrically 
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Figure  3.  Reiention  Characieristics  of  a  Modifiable  SONGS  SvTiapric  Weight 
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Figure  6.  Conceptual  View  of  Multi-layer  Artifidal  Neural  Network 
Architectxire  Incorporating  the  Single-Level  Linear  Adaptive  Neuron 
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Figure  3.  Elecmcal  Impiementation  of  the  S\'aaDtic  Weishis 
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Figure  9.  Output  and  Training  Signals  versus  Time  Cliaracterisucs  ia;  Initialised  ill  ^  Adapted 
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Figure  10.  Error  Signal  versus  Time  Characteristics 
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Figure  11. improved  Characteristics  Demonstrated  by  the  Newly 
Fabricated  SONOS  Synaptic  Weight  Elements 
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Figure  IS.Comparison  between  Old  and  New  Convergence  Time  Performance 
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Figure  14.Block  Diagram  of  the  Automated  Data  Acquisition  System 
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Figure  IG.Sample  Layout  of  the  Analog  Delay  Line 
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